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Abstract 

In this paper we extend the investigation of cooperation in some classical evolu- 
tionary games on populations were the network of interactions among individuals 
is of the scale-free type. We show that the update rule, the payoff computation 
and, to some extent the timing of the operations, have a marked influence on the 
transient dynamics and on the amount of cooperation that can be established at 
equilibrium. We also study the dynamical behavior of the populations and their 
evolutionary stability. 

1 Introduction and Previous Work 

The object of game theory is the analysis of situations where the different social actors 
have conflicting requirements and individual decisions will have a mutual influence on 
each otherfT[. In this framework, and due to their importance as simplified models of 
many common important socio-economic situations, the Prisoner's Dilemma (PD) and 
the Snowdrift (SD) games have received much attention in the literature. According to 
game theory, the PD and the SD are paradigmatic examples of games in which coopera- 
tive attitude should vanish in the PD, and should be limited to a given fraction in the SD. 
This is also the case when large populations of individuals play the game pairwise in 
a random manner and anonimously, as prescribed by evolutionary game theory J2). In 
spite of this, numerical simulations of the PD have convincingly shown that, when the 
population of players possesses a spatial structure, a certain amount of cooperation can 
emerge and remain stable. Nowak and May (3) were the first to empirically show this 
using a population structured as a square lattice where each site is a player. Standard 
evolutionary game theory is based on an infinite (or very large) population model, and 
on the random pairing of two players at each time step. This amounts to a mean-field 
spatially homogeneous model. The square grid is also spatially homogeneous but the 
absence of random mixing enables the formation of clusters of cooperators, which al- 
lows for more frequent positive encounters between cooperators than would be possible 
in the mean-field case. More recently, it has become apparent that many real networks 
are neither regular nor random graphs; instead, they have short diameters, like random 
graphs, but much higher clustering coefficients than the latter, i.e. they have more local 
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structure. These networks are collectively called small-world networks (see H for a 
recent review). Many technological, social, and biological networks are now known to 
be of this kind. Social networks, in addition, show recognizable community structure 
(5JIH. Since evolutionary PD or SD games are metaphors for conflicting social in- 
teractions, the research attention has recently shifted from random graphs and regular 
lattices towards better models of social interaction structures [7 8, Sfl fTOl . 

Recently, Santos and Pacheco J9| presented a numerical study of the evolution 
of cooperation on (static) scale-free (SF) networks for the PD and the SD games. 
Their main result was that, in contrast with what one observes in mixing populations 
or on regular lattices, much higher levels of cooperation are sustainable on this kind 
of graphs, both for the PD as well as the SD. These results are obviously interest- 
ing and encouraging for cooperation but they prompt a number of questions. First of 
all, Barabasi-Albert or correlationless configuration SF graphs [ 1 1 1 that were used in 
13 are not faithful representations of most typical social networks. In fact, although 
social interaction networks where the degree distribution can be well described by a 
power-law have been found |[T2l[T3l . several recent studies show that social networks 
in general do not have a pure power-law degree distribution function, as they often 
show signs of exponential decay of the tail of the distribution fl4l [151 . In addition, 
they usually have more clustering than pure scale-free graphs [4|. Nevertheless, model 
SF networks are a useful bounding case to study as they are closer to typical social net- 
works than other more artificial kind of graphs, such as Watts-Strogatz small worlds 
fl6l . A second aspect of social networks that is not captured by fixed graph struc- 
tures is that they are not static; rather, the number of vertices and the links between 
them continuously evolve as social actors come and go, and relationships are created 
or abandoned. Dynamical features such as these have been introduced in evolutionary 
games, among others, in ifTTl [181 [T9l l20l |2T1 . However, in this paper we only focus on 
the static aspects of the interaction networks. In other words, we make the hypothesis 
that the network is at equilibrium and that network dynamics are either absent, or their 
time scale is longer (slower) with respect to the strategy-change dynamics. This proves 
to be a useful approach, especially for social acquaintance networks. 

In the following we present a brief introduction to the games studied. This is fol- 
lowed by a discussion of the population model and of individual's payoff calculation 
scheme for the players in a complex network. Next we describe the numerical simu- 
lations and their results, including a study of evolutionary stability. We finally present 
our conclusions. 

2 Two Social Dilemmas 

Let us first recall a few elementary notions on the PD and the SD. These are two-person, 
symmetric games in which each player has two possible strategies: cooperate (C) or 
defect (D). In strategic form, also known as normal form, these games have the payoff 
bi-matrix of table Q] In this matrix, R stands for the reward the two players receive if 
they both cooperate, P is the punishment for bilateral defection, and T is the temptation, 
i.e. the payoff that a player receives if it defects, while the other cooperates. In this 
latter case, the cooperator gets the sucker's payoff S. For the PD, the payoff values are 
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Table 1: Payoff matrix for a standard two-person, two-strategies game (see text). 



ordered numerically in the following way: T > R > P > S, while in the SD game 
T > R > S > P. Defection is always the best rational individual choice in the PD - 
(D,D) is the unique Nash equilibrium and also an evolutionary stable strategy (ESS). 
Mutual cooperation would be preferable but it is a strongly dominated strategy. Thus 
the dilemma is caused by the "selfishness" of the actors. 

In the SD, when both players defect they each get the lowest payoff; (C,D) and 
(D,C) are Nash equilibria of the game in pure strategies, and there is a third equilibrium 
in mixed strategies where strategy D is played with probability 1/(2/3 — 1), and strategy 
C with probability 1 — 1/ (2/3 — 1), where /3 is another name for the temptation T, used 
in biological circles. The dilemma in this game is caused by "greed", i.e. players have 
a strong incentive to "bully" their opponent by playing D, which is harmful for both 
parties if the outcome produced is (D,D). 

3 Numerical Simulations 

The two games were simulated in [9 | on Barabasi- Albert (BA) ifTTl and configuration 
model [4 1 scale-free networks of size 10 4 over 10 4 time steps, using a discrete analogue 
of replicator dynamics equations ||2][22]. The customary rescaling of the payoff values 
was used such that there is only one independent parameter. For the PD, setting R = 1, 
P = S = 0, leaves T = b > 1 to be the only parameter (temptation). For the SD, 
T is set equal to /3 > 1, R = /3 — 1/2, S = /3 — 1, and P = 0, which makes the 
cost-to-benefit ratio of mutual cooperation r = 1/(2/3 — 1) the only parameter. For 
the sake of comparison, our simulations were done under the same conditions as in [9 1 
(10 4 players and 10 4 time steps). 

However, replicator dynamics is not the only possibility for updating the agents' 
strategies in discrete, finite populations of players using hard-wired strategies. More- 
over, in small non degree-homogeneous populations, the mathematical requirements 
behind the replicator dynamics, strictly speaking, are not satisfied ll23l . Thus, we ex- 
tended the investigation by simulating an imitate the best evolution rule according to 
which an individual i will adopt the strategy of the player with the highest payoff 
among its neighbors and itself. If a tie occurs, the winner is chosen uniformly at ran- 
dom between the best. This rule is deterministic and was the original rule used in 0. 

Concerning the calculation of an individual's payoff, there are several possibilities. 
A player's payoff may be defined as the sum (accumulated payoff) of all pair interac- 
tions with its nearest neighbors, which is the form used for instance in (9j- Another 
possibility consists in using average payoff, which is the accumulated payoff divided 
by the number of interactions. Accumulated and average payoff give the same results 
when considering degree-homogenous networks such as lattices. Accumulated payoff 
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seems more logical in degree-heterogeneous networks such as scale-free graphs since 
it reflects the very fact that players may have different numbers of neighbors in the net- 
work. Average payoff, on the other hand, smooths out the possible differences although 
it might be justified in terms of number of interactions that a player may sustain in a 
given time. For instance, an individual with many connections is likely to interact less 
often with each of its neighbors than another that has a lower number of connections. 
Also, if there is a cost to maintain a relationship, average payoff will roughly cap- 
ture this fact, while it will be hidden if one uses accumulated payoff. For the sake of 
comparing the two extreme views, here we use both accumulated and average payoff. 

Under discrete replicator dynamics rule with accumulated payoff, and using syn- 
chronous update, Santos and Pacheco 1 9 1 found that, when compared to regular lattices, 
SF networks lead to high levels of cooperation for all values of the parameters b (for 
PD) and r (for SD). These results have been reproduced by us and are shown in the 
upper half of figure Q] Cooperation is also much higher in SF graphs than what has 
been obtained for Watts-Strogatz small-world graphs QdO). When using the "imi- 
tation of the best" strategy-switching rule with synchronous update and accumulated 
payoff the results are similar, as one can see in the lower part of figure Q] although 
there is a marked fall in the high-b and high-r region with respect to replicator dynam- 
ics. However, when one lingers on the standard deviations (represented as error bars 
in the figure), one sees that the results for the imitate the best rule are noisy, with quite 
large fluctuations. Deviations are smaller for the replicator dynamics, see figureQ] The 
reason for the instability and the large fluctuations can be traced to the step function na- 
ture of the update rule, as can be seen in figure[3](a), in which 40 individual PD runs are 
plotted, all with b = 1.8. In all runs cooperation falls at the beginning, the cooperators 
then often recover but not always, as there are several runs (about 1/5 for the data used 
here) in which cooperation never recovers. On the other hand, when using replicator 
dynamics, there is still a systematic drop of cooperation at the beginning (figure[3](c)), 
nevertheless it tends to rise again in the long run, although this may happen very late in 
the simulation (see figure 13(b)). To better observe this phenomenon, we have doubled 
the number of time steps (2 x 10 4 ). 

We thus see that the results on BA SF graphs depend on the update rule, although 
the level of cooperation is still higher than what is found on regular, Watts-Strogatz, 



and random graphs 1221 
chronous update policy 



TOl . However, we wish to point out that if we use an asyn- 
l \ with the "imitate the best" rule instead of the usual syn- 
chronous one, the result is a higher level of cooperation with far less fluctuations than 
the synchronous case (compare lower parts of figures [T] and O. One might reason that 
the combination of synchronous update and of "imitate the best" is fully determinis- 
tic, which implies that particular chains of events, such as cascades of defection, will 
be amplified. Introducing stochasticity through asynchrony in the update sequence 
strongly mitigates the likelihood of such series of events. On the other hand, when 
using replicator dynamics, the lack of stochasticity in synchronous update is some- 
how compensated for by the probabilistic strategy change rule, which could explain 
the similarity of the results in this latter case (compare the upper parts of figures Q] and 



1 We use the standard uniform random choice (with replacement) of players in the population, which is a 
discrete approximation of a Poisson process. 
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Figure 1: Fraction of cooperators on SF BA networks of size 10 4 and average degree 
k = 4 with accumulated payoff and synchronous dynamics. Mean values over 50 runs. 
Upper figures: replicator dynamics; lower figures: imitation of the best. 



^respectively). 

To illustrate the influence of timing when "imitate the best" is the rule used for 
strategy update, suppose that a defector occupies the most highly connected node in 
the graph and that it is surrounded by cooperators exclusively. Then, at the next time 
step in synchronous update, all those cooperators will turn into defectors. From there, 
a wave of defection could quickly propagate through the network, leading to a state 
whereby cooperation cannot be recovered. On the other hand, when players are up- 
dated in random order, only a fraction of the neighbors will imitate the defector, at the 
same time lowering the payoff of the central defector, and thus making it less attrac- 
tive to be imitated in future encounters. This kind of process limits the propagation 
of defection and allows cooperation to establish itself and be stable. This highlights 
some shortcomings of synchronous dynamics, which is unrealistic and may give rise 
to spurious effects l24l . Our conclusion is that, although there is often no significant 
difference between synchronous and asynchronous update in evolutionary games, as it 
is the case here under replicator dynamics, the latter is to be preferred for reasons of 
generality and reliability. However, for the sake of comparison with previous results, 
in the rest of the paper we use synchronous update. 

Now we turn our attention to the assumption that a player's utility is the sum, 
i.e. the accumulated payoff of all pair interactions with its nearest neighbors. Al- 
though this appears to be a logical step to follow, we shall show that it may cause 
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Figure 2: Fraction of cooperators on SF BA networks of size 10 4 and average degree 
k = 4 with accumulated payoff and asynchronous dynamics. Mean values over 50 
runs. Upper figures: replicator dynamics; lower figures: imitation of the best. 



both conceptual and technical problems. Obviously, one would assume that if an in- 
dividual has more links to cooperators, and that the payoffs are positive quantities, 
she should earn more than another player with fewer cooperating neighbors. How- 
ever, this begs the question of how the network got there in the first place. BA SF 
graphs are incrementally built by using linear preferential attachment ifTTl . In this 
model there is no cost associated to the formation of a new link. However, although 
this model may be adequate for citation networks or, to some extent, the Web, it is 
well known that this cannot be the case in most other instances. Thus, other models 
have been proposed that take into account cost and other factors in network forma- 
tion In our case, it is as if the population would be "injected" on an already 
full-grown, topology-favorable network, while the rules of the game and other con- 
siderations necessarily should play a role in the network formation and dynamics. The 
same remarks also hold for the "configuration" SF graphs, although these networks 
are built starting from the degree distribution and a fixed number of nodes, rather than 
incrementally. Furthermore, a technical problem arises when combining replicator dy- 
namics with accumulated payoff. In infinite mixing populations, classical evolutionary 
game theory states that replicator dynamics is invariant under positive affine transfor- 
mations of payoffs with merely a possible change of time scale [2J. This invariance 
still holds in finite degree-homogenous populations. However, when different indi- 
viduals start having different degrees, things are not quite the same. Let denote 
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Figure 3: PD time series with b = 1.8; (a) imitation of the best; (b) replicator dynamics; 
(c) replicator dynamics (first 70 steps). 



a player i's aggregated payoff. Furthermore, let </>(IIj — Ilj) = (Ilj — II,-)/(.Dfe>) 
be the probability function according to which i adopts neighbor j's strategy, with 
D = max{T, R, P, S} — min{T, R, P, S} and fc> = max{ki, kj}, where k x repre- 
sents the degree of player x (9). If we now apply a positive affine transformation of 
the payoff matrix, this leads to the new aggregated payoff IIJ = alii + (3ki and hence 
0(n$-Il9 = {a\l j +f3k ] -a\l l -(3k i )/{aDk > ) = 0(I[f -Hi) + (kj -ife j )/(aJ)& > ). 
One can clearly see that using accumulated payoff does not lead to an invariance of the 
replicator dynamics under shifts of the payoff matrix. As an illustration of the violation 
of this invariance, figure [4] shows cooperation curves for the PD when applying such 
payoff transformations. 

This has several implications such as limiting the results obtained in [9| strictly to 
the studied values of b and r, and to an impossibility to rescale the payoff matrix. In a 
more recent study [25 1 Santos et al. investigated the same games in a wider parameter 
space, but still using accumulated payoff, which again makes the results non-invariant 
with respect to a positive affine transformation. Therefore, we repeated the numerical 
simulations with average payoff, i.e. the aggregated payoff obtained by one player 
divided by the number of links the player has to nearest neighbors, which, along with 
the shortcomings described above, has the advantage of leaving the replicator dynamics 
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Figure 4: Fraction of cooperation for the PD game using replicator dynamics and ac- 
cumulated payoff. A translation of the payoff matrix can produce a fall in cooperation 
(shift of —1) as well as unpredictable behaviors (shift of +5) with some runs contain- 
ing high levels of cooperation and others ending up with massive defection. Standard 
deviations are not plotted here to improve readability. 
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Figure 5: Fraction of cooperators on SF BA networks of size 10 4 with average degree 
k = 4 using average payoff and synchronous dynamics. Mean values over 50 runs. 
Upper figures: replicator dynamics; lower figures: imitation of the best. 
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invariant under positive affine transformations. 

In figure [5] we report results for the PD and SD games using average payoff with 
synchronous updating dynamics, and the same parameter set as in [9|. Looking at 
the figures, and comparing them with the results of [9| (replicated here for k = 4 in 
figure [TJ, one immediately sees that the cooperation level reached after the transient 
equilibration period is much lower, and comparable with the results found for regular 
and random graphs. This is reasonable, given that now it is as if each individual had 
the same average number of neighbors as far as its payoff is concerned. 

To reach a better understanding of the difference between accumulated and average 
payoff, we interpolated between the two extreme cases according to the formula 

3 

where d <E [0, 1], IT is the net payoff of player i, and iti j is the payoff player i obtains 
when interacting with neighbor j. One can see that, when d = we recover the 
accumulated payoff value, while d = 1 corresponds to the average payoff case. Figure 
[6] clearly shows that, as d varies from to 1, and thus the ratio varies from 1 to 1/fc, 
cooperation levels steadily decrease for all values of the temptation on the y-axis. So, 
the way in which individual payoff is computed has a large influence on cooperation 
levels that can be reached, in the average, on a given network topology. 




Figure 6: Cooperation level as a function of the parameter d of equation[T]in the PD for 
temptation values between 1 and 2. Cooperation prevails in light areas; Darker areas 
mean more defection. Results are the average of 50 runs. 



4 Evolutionary Stability 

Evolutionary stability, i.e. the resistance to invasion by mutant strategies, is an im- 
portant issue when dealing with evolutionary games Q. The effect of switching the 
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Figure 7: System stability when using accumulated payoff. For each parameter set, 100 
runs have been perturbed, but only a few individual runs are plotted here to expose the 
behaviors encountered. Upper figures: replicator dynamics; lower figures: imitation of 
the best. Left-hand figures: b = 1.8; right-hand figures: r = 0.5. 



strategy of the hub with largest connectivity in a totally cooperating population has 
been studied in [26|. Here we use a different approach to perturb the population after 
it has reached a quasi-stable state by switching the strategy of a few players having the 
strategy of the greater number. This was done for values of b G {1-2, 1.5, 1.8} and 
r G {0.2, 0.5, 0.8}. We then give the system 6000 time steps to attempt to reattain its 
initial stable state. For reasons of space, we only plot the results obtained for 6=1.8 
and r = 0.5 (see figure|7]i. Given the scale-free nature of the interaction network, in- 
troducing a small amount of random noise does not have any effect on the population 
stability. On the other hand, when cooperator hubs switch strategy (one to five in our 
study), avalanches of defection can form and propagate through the population. Under 
replicator dynamics and when using accumulated payoff, about 1 /6 of the PD runs do 
not recover the state previously attained at time step 10 4 . This fraction rises to 1 /3 for 
the SD game. With the imitation of the best rule, 1/10 of the PD and SD runs fail to 
recover from the perturbations. In contrast to accumulated payoff, average payoff does 
not allow perturbations to generate any noticeable effect, i.e. the system remains quite 
stable. 
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5 Conclusions 



In conclusion, we have deepened and extended the study presented in |9| clarifying 
the role of the updating rule and the type of payoff attributed to players. We have 
shown that the games are not invariant under linear affine tranformations when us- 
ing accumulated payoff, while average payoff does not have this problem, although it 
may artificially reduce the impact of scale-free degree networks. We have also seen 
that asynchronous update dynamics, being more likely in a system of independently 
interacting agents, by eliminating artificial effects due to the nature of synchronous 
update, may give rise to steadier quasi-equilibrium states. Moreover, we have studied 
several dynamical aspects of the evolution of the populations such as their transients 
before attaining the steady-state, and their evolutionary stability, showing that scale- 
free networks of interactions provide a quite stable environment for the emergence of 
cooperation when using accumulated payoff, except when hubs are targeted by the mu- 
tations, in which case a sizable number of runs do not recover the original state, at least 
within the simulation times allowed in our numerical experiments. 

Acknowledgments. E. Pestelacci and M. Tomassini gratefully acknowledge finan- 
cial support by the Swiss National Science Foundation under contract 200021-1 1 1816/1. 

References 

[1] R. B. Myerson. Game Theory: Analysis of Conflict. Harvard University Press, Cambridge, 
MA, 1991. 

[2] J. W. Weibull. Evolutionary Game Theory. MIT Press, Boston, MA, 1995. 

[3] M. A. Nowak and R. M. May. Evolutionary games and spatial chaos. Nature, 359:826-829, 
October 1992. 

[4] M. E. J. Newman. The structure and function of complex networks. S1AM Review, 45:167- 
256, 2003. 

[5] M. E. I. Newman and M. Girvan. Finding and evaluating community structure in networks. 
Physical Review E, 69:0261 13, 2004. 

[6] M. C. Gonzalez, P. G. Lind, and H. I. Herrmann. System of mobile agents to model social 
networks. Phys. Rev. Lett., 96:088702, 2006. 

[7] G. Abramson and M. Kuperman. Social games in a social network. Phys. Rev. E, 
63:030901, 2001. 

[8] P. Holme, A. Trusina, A. Kim, and P. Minnhagen. Prisoner's dilemma in real-world ac- 
quaintance networks: spikes and quasi-equilibria induced by the interplay between struc- 
ture and dynamics. Phys. Rev. E, 68:030901(R), 2003. 

[9] F. C. Santos and I. M. Pacheco. Scale-free networks provide a unifying framework for the 
emergence of cooperation. Phys. Rev. Lett., 95:098104, 2005. 

[10] M. Tomassini, L. Luthi, and M. Giacobini. Hawks and doves on small-world networks. 
Phys. Rev. E, 73:016132, 2006. 

[11] R. Albert and A.-L. Barabasi. Statistical mechanics of complex networks. Reviews of 
Modern Physics, 74:47-97, 2002. 



11 



[12] F. Liljeros, C. R. Edling, L. A. Amaral, H. E. Stanley, and Y. Aberg. The web of human 
sexual contacts. Nature, 411:907-908, 2001. 

[13] M. C. Gonzalez, P. G. Lind, and H. J. Herrmann. Model of mobile agents for sexual 
interaction networks. Eur. Phys. J. B, 49:371-376, 2006. 

[14] L. A. N. Amaral, A. Scala, M. Barfhelemy, and H. E. Stanley. Classes of small-world 
networks. Proc. Natl. Acad. Sci. USA, 97(21): 11 149-1 1152, 2000. 

[15] M. E. J. Newman. Scientific collaboration networks. I. network construction and funda- 
mental results. Phys. Rev E, 64:016131, 2001. 

[16] D. J. Watts and S. H. Strogatz. Collective dynamics of 'small-world' networks. Nature, 
393:440-442, 1998. 

[17] M. G. Zimmermann, V. M. Egm'luz, and M. San Miguel. Coevolution of dynamical states 
and interactions in dynamic networks. Phys. Rev. E, 69:065 102(R), 2004. 

[18] M. G. Zimmermann and V. M. Egufluz. Cooperation, social networks, and the emer- 
gence of leadership in a prisoner's dilemma with adaptive local interactions. Phys. Rev. 
E, 72:056118, 2005. 

[19] C. Biely, K. Dragosits, and S. Thurner. Prisoner's dilemma on dynamic networks under 
perfect rationality. Technical Report e-print arXiv: physics/0504190, 2005. 

[20] L. Luthi, M. Giacobini, and M. Tomassini. A minimal information prisoner's dilemma on 
evolving networks. In L. M. Rocha, editor, Artificial Life X, pages 438-444, Cambridge, 
Massachusetts, 2006. The MIT Press. 

[21] F. C. Santos, J. M. Pacheco, and T. Lenaerts. Cooperation prevails when individuals adjust 
their social ties. PLOS Comp. Biol, 2:1284-1291, 2006. 

[22] C. Hauert and M. Doebeli. Spatial structure often inhibits the evolution of cooperation in 
the snowdrift game. Nature, 428:643-646, April 2004. 

[23] M. A. Nowak, A. Sasaki, C. Taylor, and D. Fudenberg. Emergence of cooperation and 
evolutionary stability in finite populations. Nature, 428:646-650, October 2004. 

[24] B. A. Huberman and N. S. Glance. Evolutionary games and computer simulations. Proc. 
Natl. Acad. Sci. USA, 90:7716-7718, August 1993. 

[25] F. C. Santos, J. M. Pacheco, and T. Lenaerts. Evolutionary dynamics of social dilemmas in 
structured heterogeneous populations. Proc. Natl. Acad. Sci. USA, 103:3490-3494, Febru- 
ary 2006. 

[26] F. C. Santos and J. M. Pacheco. A new route to the evolution of cooperation. Journal of 
Evolutionary Biology, 19:726-733, May 2006. 



12 



1 




